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ABSTRACT 

The Fleet Numerical Facility (FNWF) 500-mb long-wave prognoses at 
latitudes 20N through 70N were analyzed statistically for possible error- 
bias. FNWF issued a correction field (verified minus prognostic) for 
each latitude circle and day under study as well as the initial height 
field 48 hours earlier. 

Fourier analyses of both the correction fields and the initial 
height fields were made. Spectral analyses of the correction fields 
indicated that more than 60% of the error lay in correction -waves 1, 2, 
and 3 at all latitudes and that this figure increased northward. 

Persistency correlations were made between the initial height fields 
and the correction fields. The results indicated a carry-over of per- 
sistency into the final 48-hr forecast, especially in the cases of 
heights which were initially considerably above or below the normal 
heights . 

The final phase of the study dealt with the question of stabiliza- 
tion of the ultra-long waves (1, 2, and 3), using the present operational 
barotropic model. The model seems to retard waves 1 and 3 at the lower 
and middle latitudes, and to move them too far at high latitudes. The 
results for wave 2 are somewhat less conclusive, but are generally in 
agreement with those for waves 1 and 3. 

The authors wish to express their sincere appreciation to Professor 
Frank L. Martin, Department of Meteorology and Oceanography, U. S. Naval 
Postgraduate School, for his suggestion of the topic and assistance in 
the preparation of this study. 
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1 . 



Introduction. 



Fleet Numerical Weather Facility (henceforth referred to as FNWF) 
issues a 48-hr "long wave" prognosis for 500 mb. This forecast is 
issued twice daily (00Z and 12Z) and is based upon a smoothed version 
of their conventional 48-hr map* The smoothing is performed by means of 
the operator 

2. = [/8 ( 2, + Z x + Z 3 +Z+) + >/i 2X 

which reduces the amplitudes of the short waves while leaving that of 
the long waves relatively unchanged. 

FNWF provided the authors with 36 daily values of the forecast and 
verifying heights at 10° longitude intervals along each of the six 
latitude circles 20 through 70N. In addition, they also provided for 
each case similar data from the analyzed chart which served as the 
initial-data chart 48 hours earlier. The authors then examined, by 
statistical means, the hypothesis that the correction field (i.e., the 
set of correction values, as a function of latitude L and longitude t~ 8 
necessary to convert the forecast long-wave chart to the verifying 
chart) was at least partially determined or biased by the original 
analysis used in starting the 500-mb prognosis. 
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2 * 



Data Sources . 



As mentioned in the previous section, the primary data-charts used 
in this study were the analyzed 500-mb long-wave charts at 0000 GCT 
together with the "correction" long -wave charts timed 48 hours later . 

The data were restricted to the month of December 1962, so that it was 
possible to make a total of 29 comparisons between initial and correction 
charts . 
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3. Statistical Techniques. 



(a) Spectral Analysis of the Correction Field 

Before any relationships between the correction and initial 
fields were sought, it was decided to investigate the spectral character* 
istics of the correction field. This was done by expressing the 
'Correction" long-wave field, denoted ^(fr) > as the sum of twelve 

harmonic waves by Fourier analysis: 

Y(r) = Vl °<„ + Z ( °<s cos ' a "V 5 - +& s Sln (1) 

where S * wave number (1 through 12). 

D = 36 is the number of data points on each latitude circle 

r = 2, . . . , f) -1 is a distance scale numbered in equally - 

spaced data points measured westward from Greenwich. 

Then the spectral contribution of the £fh harmonic in Y(r) is given by 



'/* ( + $:) or h ci 

The last statement implies that (2) has been expressed in the equivalent 
form 



(3) 



Y(r) = % ^ + c s cos l ^ 

where ^ is the phase angle. It has been found more convenient in 

this study to use phase "shift" ^ rather than phase angle. This 

•S 

is done by expressing Y(r) as 



Y(r) ~ +■ c s (4 

The advantage of using (4) over (3) is that it enables one to 
keep track of the same identifiable feature (for example, the ridge 
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closest to Greenwich) for each wave number 

Using the CDC-1604 digital computer, the Fourier series was 
derived for each latitude and day. Use was made of the FORTRAN program 
"Periodogram Analysis" written by R. R. Hilleary [3] of the U. S. Naval 
Postgraduate School Computer Center staff. The program-output gave for 
each wave number S , each latitude and day minder consideration, the 
following : 



(1) Amplitude and spectral value 

(2) Phase angle and phase shift $ s - fy/s 

(3) Mean value of X and the total variance of VO'") 

(4) Percent contribution of the £ fh wave to total variance 

of V(r) 

In this subsection, attention was focused upon item 4, that is, 
relative spectral contributions of the various correction-field waves. 

In order to obtain representative values, the 29 daily percent contribu- 
tions by waves were averaged to give the December 1962 mean relative- 
percentage contributions to variance of the correction field. The 
resulting percentages are displayed in table 1, both as a function of 
latitude and wave number. 

In addition, any wave listed in table 1 which contributed, in 
the mean, more than 10% of the variance at its indicated latitude has 
been subjected to frequency analysis using histograms. The waves so 
examined are indicated in table 1 by an asterisk following the appro- 
priate percentage contribution. Waves 1 to 4 fall into this category at 
latitudes 20 through 50, while waves 1 to 3 contribute more than 10% at 
all latitudes 20 through 70. Wave 6 contributes more than 10% at 
latitude 60, but all other values of all waves fall below 10%. 
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Table 1. 


Mean 

the 


explained 

correction 


percentage contribution to the total 
field by wave number and latitude. 


variance < 






20N 


30N 


40N 


5 ON 


60N 


70N 


Wave No. 


i 


14.5* 


21.2* 


20.9* 


16.7* 


30.3* 


35 . 2* 


Wave No. 


2 


17.7* 


16.9* 


19.9* 


25.3* 


26.6* 


32.2* 


Wave No. 


3 


28.9* 


22.1 


18.9* 


14.5* 


15.9* 


16.2* 


Wave No. 


4 


16.6* 


12.8* 


12.1* 


10.9* 


8.1 


9.5 


Wave No. 


5 


6.5 


6.2 


5.9 


6.8 


8.4 


3.3 


Wave No. 


6 


3.9 


4.7 


7.2 


10.1* 


4.7 


1.3 


Wave No. 


7 


3.3 


4.5 


4.9 


7.3 


2.8 


1.3 


Wave No. 


8 


2.0 


2.9 


3.0 


2.3 


1.3 


0.4 


Wave No. 


9 


1.8 


3.2 


2.6 


1 . 8 


0.6 


0.3 


Wave No. 


10 


1.5 


2.1 


1.4 


0.9 


0.4 


0.1 


Wave No. 


11 


0.7 


1.4 


1.2 


0.9 


0.3 


0.1 


Wave No. 


12 


0.9 


0.7 


0.7 


0.7 


0.1 


0.0 


Total 

Explained 

Variance 


98.3 


98.7 


98.7 


98.2 


99.5 


99.9 



* Denotes wave contributing more than 10% to the correction field at 
the indicated latitude. 

The histograms drawn are shown in the Appendix as figures 1-6 
(in order of increasing latitude) for wave 1, figures 7-12 for wave 2, 
figures 13-18 for wave 3, figures 19-22 for wave 4, and figure 23 for 
wave 6. Note that all these diagrams present in the upper half a fre- 
quency distribution with regard to wave amplitude* and in the lower half 
a frequency distribution of the phase shift giving the correction-wave 
ridge closest to Greenwich . 
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As an example of the type of conclusion which may be drawn 
from these histograms, one may compare figure 1 with figure 6, both 
dealing with wave 1, Figure 1 indicates that wave 1 has a small mean 
amplitude (66 feet) and a mean ridge position at -15W longitude** By 
contrast, figure 6 has a mean amplitude of 155 feet and a mean correction 
phase shift of -7W longitude** However, note that frequency distribu- 
tions of the phase shift appear to be random in both cases, though not a 
Gaussian distribution in either case. 

The same general latitudinal variation occurs with wave 2 in 
passing from latitudes 20N to 70N. At 20N there is only a small mean 
amplitude (about 60 feet) and a random distribution of phase shifts 
centered around a mean of SOW, whereas at latitude 70N the wave 2 cor- 
rection field had an amplitude of 150 feet and a phase shift of 15W. 

This means that the trough in the wave 2 correction field at 70N would 
be located at 105W in December. 

The mean values of amplitudes and phase shifts in figures 1-23 
have been computed and are displayed by each histogram* It is not known 
whether such means have any real significance* Of course it is possible 
that the correction field histograms depicted show real prognostic 
anomalies which the barotropic forecast model cannot resolve, and are 
applicable to December in general. One approach in determining whether 
these correction -waves have real statistical significance is to test the 
cross-correlation between correction and harmonic waves by latitude and 
wave number 



* 



r(c 



$ ) 









(5) 






Note that all FNWF long-wave information was printed in terms of 
degrees west longitude so that the two phases mentioned above are 15 
and 7 east longitude*. 
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for significance* The expression on the right side of (5) is given by 
Kahn [4l * Actually this test was not performed, mainly because a 
simpler approach, described below, was applied* 

(b) Correlation Investigation of Anomalies 

A FORTRAN data-sorting program was written for each latitude 
and all of the 29 days* This program sorted out abnormally high and low 
contour values in the 29-day sample, and a correlation subroutine com- 
puted the correlation between the initial contour height at the latitude 
longitude intersection ( r ) and the f ©recast-correction value verify 
ing two days later* The sorting program was designed to divide the 
29 x 36 value sample at each latitude into classes as follows:: 

(I) Those heights more than one standard deviation above the 
sample mean height (approximately 16% if the heights are 
normally distributed)* 

(II) Those heights smaller than the mean by at least one 
standard deviation* 

(III) The remaining 68% of the heights between extremes (I) 
and (II)* 

The linear correlations obtained for each latitude and class 



are listed in table 2 below* 

Table 2* Linear correlations between initial height fields and 48-hr 
verification-correction values* 







20N 


30N 


40N 


50N 


60N 


TON 


Class 


i 


0.08 


-0.01 


0.02 


0.16 


0 06 


-0.41 


Class 


ii 


-0.29 


-0.17 


-0.13 


0.09 


0.15 


-0.22 


Class 


hi 


-0.03 


-0.13 


0.07 


-0.12 


-0.03 


-0.13 



If we make the assumption that half of the data-sample pairs are 
uncorrelated, significance tests for each class will be based upon the 






reduced sample sizes N = 84, 84, and 354, respectively, for classes I, 
II and IIIo For these sample sizes, the following critical values £ 
of the linear correlation coefficient are applicable (see Dixon and 
Massey [1957, p. 468] at the 997o confidence level )? 



(I) 


N . 84 




s 0.27 


(II) 


N s 84 


£ 


s 0.27 


(III) 


N = 354 


£ 


s 0.13 



A significant negative correlation between correction values 
and the 500-mb heights two days earlier indicates that initially above- 
average heights tend to perpetuate themselves unduly in the prognoses 
two days later. This effect of persistence is most marked with classes I 
and II, which are already considerably above or below normal. The most 
significant correlations in these classes correspond to (I, 70N) and 
(II, 20N) with r = -0.41 and -0.29, respectively. Generally, class II 
shows a preponderance of near-significant negative values (see lati- 
tudes 30, 40, and 70N in addition to latitude 20N) , which tend to bear 
out the comments made previously with respect to the effects of per- 
sistence . 

Conversely, class III, even with the large number of "inde- 
pendent" pairs, has no definitely significant correlation coefficients. 
These results suggest that the FNWF barotropic model appears to handle 
weak ridges and troughs quite well. 

(c) Persistence of the Ultra -Long . Waves 

As mentioned in section 3(a), the frequency distributions of 
correction-wave phase shifts shown in figures 1-23 cannot be 

expected to apply for another December forecast series. Moreover, while 
the frequency distributions do not appear to be normally distributed, 
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there is an element of randomness about them in terms of values of 
relative to Greenwich. Hence, a relative test concerning the efficacy 
of the correction-wave phase shift has been devised. In this test, 

attention is focused only on the harmonic waves S s 1, 2, 3, called 
11 ultra-long^waves . 

The barotropic model employed by FNWF makes use of a long-wave 
"stabilization" term called the Helmholtz correction term. This term is 
due to the initial work of Wolff [6] and Cressman [7], and was devised 
specifically to keep waves S s 1, 2, 3 stationary, whereas the previous 
version of the barotropic model permitted spurious movement of these 
waves. For the purpose of testing whether these waves were stabilized , 
the variable 

^ ( 6 > 



which may be termed "relative phase shift," was computed at each latitude 
for each day and wave numbers S s 1, 2, 3. In (6) the symbol ^ ^ 
represents the phase of the Sth Fourier wave derived from the analysis 
two days previous to the long -wave prognosis. In (6) both and 

denote the meridians of the forecast-correction and initial waves closest 
to the Greenwich meridian. It was considered, in this section of the 
work, that if A $ lay in the range -5 to +5 for the 48-hr forecast 
period, the wave was actually stabilized and such values of were 

termed "zero" for the purposes of this investigation. 

Values of i*§i larger than 5° longitude were considered to 
be exact, and the number of positive, negative and "zero" values occurring 
in each sample of 29 cases are listed in table 3 for each latitude. In 
addition, mean values of A ^ and mean magnitudes have beer listed. 
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Table 3. Some statistics on the behavior of relative phase shift. 







Mean 


Mean 




of Ai 




Signifi- 






A $ 


|a?| 


Signs 




cance 




Latitude 


Positive 


Negative 


Zero 


Level 




20N 


-12.9° 


120.4° 


17 


12 


0 


ic 




30N 


-71.3° 


104.7° 


6 


22 


1 


0,2% 


o 


40N 


-64.1° 


76.4° 


2 


25 


1 


0.2% 




5 ON 


-57.6° 


77.4° 


6 


23 


0 


0.2% 


<D 

> 


60N 


+ 6.5° 


116.3° 


13 


15 


1 


N 


£ 

22 


70N 


+19.0° 


104.7° 


14 


15 


0 


N 




20N 


+18.7° 


32.3° 


22 


5 


2 


* 


CM 


30N 


- 7.5° 


44.5° 


11 


14 


4 


N 


O 


40N 


0.0° 


34.8° 


11 


15 


3 


N 


S3 


SON 


- 6.9° 


32.7° 


12 


16 


1 


N 


0) 

> 


60N 


-23.4° 


50.1° 


8 


20 


1 


1.2% 


c0 

22 


7 ON 


- 3 . 2° 


47.5° 


13 


14 


2 


N 




20N 


+13.6° 


49.2° 


17 


10 


2 


ic 


cn 


30N 


-13.6° 


37.6° 


8 


19 


2 


2.6% 


o 


40N 


-15 . 6° 


28.8° 


6 


22 


1 


0.2% 


22 


SON 


+ 1.5° 


25.8° 


13 


14 


2 


N 


<u 

> 


60N 


+ 5 . 8° 


29.9° 


12 


13 


4 


N 


CO 

& 


7 ON 


+ 5.9° 


29.9° 


17 


10 


2 


12.4% 




* Latitude 20N not 


included 


in test. 









N Indicates the distribution of signs is not significantly dif- 
ferent from a "chance" distribution 



The first of these gives a ready answer to the question of the 48~hr dis- 
placement of the ultra-long correction-waves in degrees west longitude. 

From table 3a, it appears (se H e column 1, excluding latitude 20N) that: 

Wave 1 correction field is over -displaced to the east in latitudes 
30-50N, but is displaced westward at 60 and 70N. 

Wave 2 is slightly over-displaced at all latitudes. 

Wave 3 is over-displaced at low latitudes (30, 40N) , but negatively 
displaced at high latitudes. 

Significance levels have not been ascribed to mean ^ values in 
table 3, so that the conclusions on displacement proposed just above are 
only tentative and subject to further testing, for example using the £ -test. 
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A test has also been applied to the signs contained in the sign 
column of table 3c The significance levels have been determined (using 
table 10a, p 418 of Dixon and Massey, 1957) and the implication is that, 
at these levels, the combination of positive and negative signs could 
only occur with a probability equal to the quoted level in table 3, 
assuming either type of sign were equally probable* 

The conclusions to be drawn from the sign-signif icance tests as 
applied to table 3 are as follows i 

(a) For wave 1 there is strong evidence that the ultra-long 
waves require displacement to the east relative to the 
prognostic location at latitudes 30, 40, and 50N in a 
majority of the cases* This result is in agreement with 
that noted earlier in connection with the value of the 
mean A • 

(b) For wave 2 at higher latitudes (60N) the barotropic model 
gives under-displacement compared to observation* 

(c) For wave 3 there is a curious but consistent pattern at 
latitudes 30 and 40N. The barotropic model gives con- 
sistent under-displacement as compared to observation* 

The opposite conclusion at latitude 70N appears to have 
moderate significance, that is, a 12*4% "chance-probability a 10 
At latitudes 50 and 60N, the distribution of signs appears 

to be completely random* 
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4. 



Conclusions . 



These statistical studies of the FNWF 500-mb prognoses have shown a 
number of significant results . Some of these results are given below. 

(a) More than 607* of the variability of the 48-hr correction field 
is associated with correction waves 1, 2, 3 with the former 
taking on a consistently greater proportion with increasing 
latitude. 

(b) There is a tendency for above and below normal heights to be 
perpetuated in the subsequent 48-hr forecast. 

(c) Generally in the low latitudes (30 through 50N) the ultra-long 
waves must be displaced to the east of their prognostic 
location, while the reverse seems to be true at latitudes 60 
and 70N. 
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APPENDIX 



Histograms of amplitude and phase shift for correction waves 
accounting for more than 107. of the latitudinal variance of the wave 
(figures 1 through 23). For descriptions of these figures and appro- 
priate page numbers, refer to the list of figures on page iv. 
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Figure 3 
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